Imaging apparatus and control method

ABSTRACT

A second area corresponding to a first pupil and included in the first area, is designated. The driving of a focus lens is controlled based on an image signal corresponding to the second area. The designation is canceled in a case where a first area becomes no longer detected.

BACKGROUND Field of the Disclosure

The present disclosure relates to an imaging apparatus that canautomatically focus on a pupil.

Description of the Related Art

In a digital camera using an image sensor, there is known a techniquefor detecting a face of a person from image data obtained from the imagesensor, and bringing a pupil into focus in a case where the pupil isdetected from an area corresponding to the face in the image data.

Japanese Patent Application Laid-Open No. 2015-96961 discusses bringinga pupil into focus in a mode where a user designates bringing either theright or left pupil into focus.

According to Japanese Patent Application Laid-Open No. 2015-96961,either the right or left pupil desired by the user can be brought intofocus. However, Japanese Patent Application Laid-Open No. 2015-96961still has an issue in terms of usability. For example, in a case wherethe user has designated either the right or left pupil, the designatedpupil is brought into focus even if the user desires to bring the pupildifferent from the designated pupil into focus unless the user performsthe designation again. Further, the same one of the right and leftpupils is brought into focus, for example, regardless of whether theface of the same person is detected.

SUMMARY

The present disclosure is directed to a technique for focusing on apupil exactly as intended by a user, as compared with a conventionaltechnique.

According to an aspect of the present invention, an image capturingapparatus includes at least one processor configured to performoperation of: a first detection unit configured to detect a first areabased on an image signal output from an image sensor, a second detectionunit configured to detect a second area corresponding to a first pupil,the second area being included in the first area, a designation unitconfigured to designate the second area detected by the second detectionunit, and a control unit configured to control driving of a focus lensbased on an image signal corresponding to the second area, wherein thedesignation unit cancels designation of the second area in a case wherethe first area becomes no longer detected by the first detection unit.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration for implementing anexemplary embodiment of the present invention.

FIG. 2 is a flowchart illustrating operation according to a firstexemplary embodiment of the present invention.

FIG. 3 is a flowchart illustrating a method for automatic pupilselection.

FIGS. 4A, 4B, 4C, 4D, and 4E each illustrate an example of transition ofdisplay of a face display frame or a pupil display frame according tothe first exemplary embodiment.

FIG. 5 is a flowchart illustrating operation according to a secondexemplary embodiment of the present invention.

FIG. 6 is a flowchart about a face detection resuming method based onface authentication operation.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described withreference to the attached drawings. In each of the exemplaryembodiments, an imaging apparatus having a pupil detection function isused as an example. As the imaging apparatus having the pupil detectionfunction, a video camera, a digital camera, a silver-halide stillcamera, and a portable device equipped with a camera function such as asmartphone may each form an aspect of the present invention.

FIG. 1 is a diagram illustrating a configuration example of an imagingapparatus according to a first exemplary embodiment of the presentinvention. FIG. 1 illustrates a configuration of a mirrorless camera(hereinafter referred to as the camera) equipped with a pupil auto-focus(AF) function.

An interchangeable lens 100 is one of optical devices attachable to acamera main body 120. The interchangeable lens 100 includes an imagecapturing lens unit 101. The image capturing lens unit 101 includes amain image capturing optical system 102, a diaphragm 103 that performslight-amount adjustment, and a focus lens group 104 that performs focusadjustment.

A lens system control microcomputer (hereinafter referred to as the lenscontrol unit) 111 includes a diaphragm control unit 112 that controlsoperation of the diaphragm 103, and a focus lens control unit 113 thatcontrols operation (also referred to as driving) of the focus lens group104. The focus lens control unit 113 drives the focus lens group 104 inan optical axis direction of the image capturing lens unit 101 based onfocus-lens driving information acquired from the camera main body 120,so that the focus adjustment of the camera is performed.

The focus lens group 104 may have a plurality of focus lenses or asingle focus lens. Here, a fixed focal length lens is illustrated as anexample of the interchangeable lens 100 to simplify the drawing.However, the interchangeable lens 100 may be a lens that can change afocal length (i.e., zoom lens). In a case of the zoom lens, the focuslens control unit 113 acquires focal-length information from an outputof an encoder that detects a zoom-lens position. Further, in a case of alens equipped with a camera-shake correction function, the focus lenscontrol unit 113 also controls a shift lens group for camera-shakecorrection.

The camera main body 120 includes a shutter 121 to be used for exposurecontrol, and an image sensor 122 such as a complementary metal oxidesemiconductor (CMOS) sensor. An imaging signal output by the imagesensor 122 is processed by an analog signal processing circuit 123, andthen is transmitted to a camera signal processing circuit 124.

A camera system control microcomputer (hereinafter referred to as thecamera control unit) 131 controls the entire imaging apparatus. Forexample, the camera control unit 131 drives the shutter 121 bycontrolling driving of a motor for shutter driving (not illustrated). Amemory card 125 is a recording medium that records data of a capturedimage. A release switch 181 is a member to be operated by a user. Apressed-state of the release switch 181 is transmitted to the cameracontrol unit 131. Based on the state, the captured image is stored inthe memory card 125.

An image display unit 171 includes a display device such as a liquidcrystal display (LCD) panel for the user to monitor an image to becaptured with the camera and to display the captured image. A touchpanel 172 is an operation unit that enables the user to designatecoordinates on the image display unit 171 using a finger or a stylus.The touch panel 172 can be integral with the image display unit 171. Forexample, the touch panel 172 may be of a built-in type (in-cell type)that is built in a display screen of the image display unit 171. In thiscase, the touch panel 172 is configured to have light transmittance thatdoes not interfere with display of the image display unit 171. Further,display coordinates on the touch panel 172 and input coordinates on theimage display unit 171 are in correspondence with each other. As aresult, a graphical user interface (GUI) can be configured that enablesthe user to feel as if the user is directly operating a screen on theimage display unit 171. The camera control unit 131 manages the state ofan operation performed on the touch panel 172.

The camera main body 120 includes a mount contact portion 161 on a mountsurface with the interchangeable lens 100. The mount contact portion 161is a communication terminal for communicating with the interchangeablelens 100. The interchangeable lens 100 includes a mount contact portion114 on a mount surface with the camera main body 120. The mount contactportion 114 is a communication terminal for communicating with thecamera main body 120.

The lens control unit 111 and the camera control unit 131 controlcommunication so that serial communication is performed at predeterminedtiming via the mount contact portions 114 and 161. Through thiscommunication, information such as the focus-lens driving informationand diaphragm driving information is transmitted from the camera controlunit 131 to the lens control unit 111, and optical information such as afocal length is transmitted from the lens control unit 111 to the cameracontrol unit 131.

The camera signal processing circuit 124 includes a face informationdetection unit 141, and further includes an organ information detectionunit 142. The organ information detection unit 142 detects organinformation about organs such as a pupil and a mouth from faceinformation detected by the face information detection unit 141. Adetection result obtained by each of the face information detection unit141 and the organ information detection unit 142 is transmitted to thecamera control unit 131.

The camera control unit 131 has an automatic pupil selection unit 150and a display frame setting unit 151 as blocks related to the presentinvention. The automatic pupil selection unit 150 automatically selectsa target pupil from detected face information. The display frame settingunit 151 sets a detection frame to be displayed by the image displayunit 171 in correspondence with the detected face or pupil information.The camera control unit 131 further includes a pupil designation anddesignation cancelation unit 152 that designates a user-designated pupilas a pupil to be continuously detected, or cancels the designation basedon an operation by the user. The camera control unit 131 furtherincludes a storage unit 153 and an AF target object setting unit 154.The storage unit 153 stores a pupil or face designated based onoperation by the user. The AF target object setting unit 154 notifiesthe selected or designated pupil or face to a focus detection unit 155as an object to be brought into focus (also referred to as the targetobject). These units operate based on outputs from the face informationdetection unit 141 and the organ information detection unit 142. Thefocus detection unit 155 performs focus detection processing based on animage signal corresponding to the object notified by the AF targetobject setting unit 154 as the object to be brought into focus. Thefocus detection unit 155 performs the focus detection processing by, forexample, a known phase-difference detection method or a contrastdetection method. In the case of the phase-difference detection method,the focus detection unit 155 performs processing for calculating animage displacement amount determined by a correlation computation for apair of image signals having parallax or processing for calculating adefocus amount by further converting the image displacement amount intothe defocus amount, as the focus detection processing. The defocusamount can be further converted into a focus-lens driving amount inconsideration of, for example, sensitivity at the time of driving theinterchangeable lens 100. The camera control unit 131 transmits, to thelens control unit 111, the focus detection result (the imagedisplacement amount or the defocus amount) obtained by the focusdetection unit 155 or the focus-lens driving amount calculated based onthe focus detection result. The focus lens control unit 113 controlsdriving of the focus lens based on the focus-lens driving informationreceived from the camera control unit 131. In other words, the cameracontrol unit 131 controls driving of the focus lens via the focus lenscontrol unit 113.

<Pupil Designation Processing>

FIG. 2 is a flowchart illustrating operation related to the firstexemplary embodiment of the present invention, in the camera controlunit 131. Processing in FIG. 2 is described as processing to beperformed each time an image frame is generated; however, the processingcan be modified as appropriate.

The operation of the first exemplary embodiment of the present inventionwill be described in detail below with reference to FIGS. 1 and 2.

First, the face information detection unit 141 performs processing (notillustrated) for detecting an area corresponding to a face(corresponding to a first area) from an image signal obtained byimaging. Further, the organ information detection unit 142 performsprocessing for detecting an area included in the area corresponding tothe face detected by the face information detection unit 141 andcorresponding to a pupil, from this image signal.

In step S201, the camera control unit 131 determines whether a face isdetected by the face information detection unit 141 in the imageobtained by imaging. If the face is detected (YES in step S201), theoperation proceeds to step S202. In step S202, the camera control unit131 sets a single face frame display flag.

Next, in step S203, the camera control unit 131 determines whether apupil is detected by the organ information detection unit 142.

If the pupil is detected (YES in step S203), the operation proceeds tostep S204. In step S204, the camera control unit 131 determines whethera pupil designating flag is set.

If the pupil designating flag is not set (NO in step S204), theoperation proceeds to step S205. In step S205, the camera control unit131 determines whether either the right or left pupil is designatedthrough on an operation performed on the touch panel 172. Morespecifically, the camera control unit 131 determines whether the pupildesignation and designation cancelation unit 152 is in a state ofdesignating either the right or left pupil based on an operation by theuser. The user can designate either the right or left pupil by touchingthe pupil that the user desires to bring into focus on the touch panel172. In a case where a touch operation for pupil designation isperformed, the camera control unit 131 determines that the pupil isselected.

If the pupil is not yet designated (NO in step S205), the operationproceeds to step S206. In step S206, the camera control unit 131executes automatic pupil selection processing. The automatic pupilselection processing will be described in detail below with reference toFIG. 3.

If either the right or left pupil is automatically selected in stepS206, the operation proceeds to step S207. In step S207, the cameracontrol unit 131 sets a single pupil frame display flag at a position ofthe automatically selected pupil, and the operation proceeds to stepS216.

On the other hand, if the pupil is designated (YES in step S205), theoperation proceeds to step S208. In step S208, the camera control unit131 performs pupil-designation storage processing. In this step, thecamera control unit 131 performs processing for storing informationindicating whether the designated pupil is of the left eye or the righteye in the storage unit 153. In addition, the camera control unit 131resets parameters related to automatic selection. The parameters relatedto the automatic selection in the present exemplary embodiment include acounter for changing between the right and left pupils as anautomatically selected pupil, and an automatically selecting pupil flag.These parameters are targets for resetting.

Then, in step S209, the camera control unit 131 sets the pupildesignating flag, and the operation proceeds to step S211.

Here, the processing will be described referring back to step S204. Thestate where the pupil designating flag is set in step S204 indicates astate where either the right or left pupil designated by the user isstored in the storage unit 153. If the pupil designating flag is set(YES in step S204), i.e., if either the right or left pupil isdesignated by the user, the operation proceeds to step S210. In stepS210, the camera control unit 131 determines whether the pupil detectedin step S203 matches the pupil designated on the touch panel 172(corresponding to a first pupil) and stored in step S208 (i.e., whetherthe designated pupil is detected). If the designated pupil is detected(second area corresponding to the first pupil is detected) (YES in stepS210), the operation proceeds to step S211.

In step S211, since the designated pupil is present in an angle of view,the camera control unit 131 sets a double pupil frame display flagindicating that the pupil is designated at a position of the designatedpupil. Then, the processing proceeds to step S216.

On the other hand, if the pupil is not detected by the organ informationdetection unit 142 (NO in step S203), the operation proceeds to stepS212. In step S212, the camera control unit 131 determines whether thepupil designating flag is set. If the pupil designating flag is set (YESin step S212), the current state is a state where a pupil is notdetected but the face is still detected, and it is highly possibilitythat the designated pupil is detected again. Thus, in step S213, thecamera control unit 131 sets a double face frame display flag toindicate that the pupil designation continues, and then the operationproceeds to step S216.

If the pupil designating flag is not set (NO in step S212), the currentstate is a state where the pupil is not designated and is not detected.Then, the processing directly proceeds to step S216.

If the camera control unit 131 determines that the pupil determined tobe detected in step S210 is a pupil (corresponding to a second pupil)different from the designated pupil (i.e., a third area corresponding tothe second pupil is detected) (NO in step S210), the operation proceedsto step S213. In step S213, the camera control unit 131 sets the doubleface frame display flag, and then the operation proceeds to step S216.

Setting the double face frame display flag in step S213 is acharacterizing feature of the first exemplary embodiment of the presentinvention. This is a state where the pupil designating flag is notcleared, i.e., the designated one of the right and left pupils remainsstored, as long as a face is found even if the designated pupil is notfound. In addition, the fact that the pupil designation is not canceledis clearly displayed for the user, by displaying a face in a doubledisplay frame.

If the face is not detected (NO in step S201), the operation proceeds tostep S214. In step S214, the camera control unit 131 clears the pupildesignating flag, thereby canceling the pupil designation. Then, in stepS215, the camera control unit 131 sets a flag for displaying multiplepoint display frames that indicates a state where neither a face nor apupil is detected instead of displaying a face display frame or a pupildisplay frame. Then, the operation proceeds to step S216.

In step S216, the focus detection unit 155 performs focus detection.Based on a result of the focus detection, the lens control unit 111controls driving of the focus lens. Here, if the camera control unit 131has determined that the face is not detected in step S201, the focusdetection is performed based on, for example, an image signalcorresponding to each of the multiple point frames. In this process, forexample, a frame is selected by a known method such as a method ofselecting the closest focus detection result from among the focusdetection results of the multiple point frames. Driving of the focuslens is controlled based on the focus detection result corresponding tothe selected frame. On the other hand, if the camera control unit 131has determined that a pupil is not detected in step S203, the focusdetection is performed by using an image signal of an area correspondingto the face. In other cases, the focus detection is performed by usingan image signal of an area corresponding to a pupil stored as the pupilselected in step S206 or designated in step S208.

In step S216, the display frame setting unit 151 sets the frame such asthe face display frame or the pupil display frame based on a conditionof the set frame display flag, and the image display unit 171 executesdisplay based on this setting. Then, the processing proceeds to stepS217.

In step S217, the camera control unit 131 clears all the frame displayflags set based on the detection result in a current frame. Accordingly,a state of the display frame to be newly displayed in a subsequent frameis determined based on a detection result in the subsequent frame.

<Automatic Pupil Selection Processing>

FIG. 3 is a flowchart illustrating the automatic pupil selectionprocessing to be executed by the camera control unit 131 in step S206.

In step S301, the camera control unit 131 determines whether both eyesare detected by the organ information detection unit 142.

If both eyes are detected (YES in step S301), the operation proceeds tostep S302. In step S302, the automatic pupil selection unit 150 selectseither the right or left pupil based on an orientation and a position ofthe face detected by the face information detection unit 141. Forexample, in a case where the face faces forward, the pupil closer to thecenter of the angle of view is selected, and in a case where the facefaces either leftward or rightward, the pupil closer to the camera isselected.

If only the pupil of one eye is detected (NO in step S301), theoperation proceeds to step S303. In step S303, the automatic pupilselection unit 150 selects the detected pupil.

Next, in step S304, the camera control unit 131 determines whether thepupil is currently in an automatically selected state. If the pupil isnot in the automatically selected state (NO in step S304), the operationproceeds to step S312. Processing in and after step S312 is performed ina case where the automatic pupil selection processing is executed forthe first time.

In step S312, the camera control unit 131 sets the pupil selected instep S302 or step S303 as a target object.

In step S313, the camera control unit 131 stores information indicatingwhich one of the right and left pupils is selected as the target objectthis time in the storage unit 153. Then, in step S314, the cameracontrol unit 131 sets the automatically selecting pupil flag.

On the other hand, if the pupil of either of the eyes is alreadyselected as the target object (YES in step S304), the operation proceedsto step S305. In step S305, the camera control unit 131 determineswhether the pupil selected in step S302 or step S303 is the same pupilas a pupil previously set as the target object. If the selected pupil isnot the same pupil as the pupil previously selected (NO in step S305),the operation proceeds to step S306. In and after step S306, the cameracontrol unit 131 performs processing for determining whether to changethe pupil set as the target object to the other pupil.

In step S306, the camera control unit 131 increments a counter.

In step S307, the camera control unit 131 determines whether the counterindicates a value greater than or equal to a predetermined value. If thecounter indicates the value greater than or equal to the predeterminedvalue (YES in step S307), the operation proceeds to step S308. In stepS308, the camera control unit 131 causes the automatic pupil selectionunit 150 to change the target pupil from the current pupil to the otherpupil. Then, in step S309, the camera control unit 131 controls thestorage unit 153 to store the new target pupil therein so that thestored pupil is updated. Then, in step S310, the camera control unit 131clears the counter and prepares for the next pupil changing processing.

If the counter does not indicate the value that is greater than or equalto the predetermined value (NO in step S307), the operation proceeds tostep S311. In step S311, the camera control unit 131 sets the same pupilas the previously set pupil as the target object. Alternatively, in acase where the same pupil as the previously set pupil is not detected,the camera control unit 131 sets the same position as the position ofthe pupil previously set as the target object as the target object, andsuspends the pupil changing processing. This processing based on thecounter is performed to prevent display from becoming unnatural due tosuccessive changing of the selected pupil.

If the selected pupil is the same pupil as the previously selected pupil(YES in step S305), the operation proceeds to step S310. In step S310,the camera control unit 131 clears the counter.

As described above, in the automatic pupil selection processing, forexample, in the case where the face faces forward, the pupil closer tothe center in the angle of view is selected. In the case where the facefaces either leftward or rightward instead of forward, the pupil closerto the camera is selected. Thus, the setting is made so as to enableeasy focusing. Furthermore, in a situation where it is easier to bringthe unselected pupil into focus, the operation for automaticallychanging the pupil to be selected is performed.

<Display Examples of Display Frame>

FIGS. 4A, 4B, 4C, 4D, and 4E each illustrate an example of the displayframe illustrated in the flowchart in FIG. 2, and each illustrate anexample of the face display frame and/or the pupil display framedisplayed by the image display unit 171.

FIG. 4A illustrates the example of the display frame at the time of thepupil automatic selection. This example is displayed in a case where theoperation proceeds to step S202, step S203, step S204, step S205, stepS206, step S207, and step S216.

FIG. 4B illustrates the example of the display frame at the time of thepupil designation. In this example, the single face display frame andthe double pupil display frame are displayed. This example is displayedin a case where the operation proceeds to step S205, step S208, stepS209, step S211, and step S216, or to step S204, step S210, step S211,and step S216.

FIG. 4C illustrates the example of the display frame in a case where theoperation proceeds to step S210, step S213, and step S216, or to stepS203, step S212, step S213, and step S216. This example is displayed ina case where a pupil is designated but the designated pupil is notdetected.

FIG. 4D illustrates the display example of the multiple point framesdisplay in a case where the operation proceeds to step S214, step S215,and step S216, i.e., in a case where a face is not detected.

In the present exemplary embodiment, only the face display frame isdisplayed in a case where a pupil other than the designated pupil isdetected, but the present exemplary embodiment is not limited to such aconfiguration. In a case where only the pupil other than the designatedpupil is detected, the pupil other than the designated pupil may beprovided with a display frame that clearly indicates that the pupil isbeing designated by the user. For example, the pupil display frame maybe displayed as a single frame, and the face display frame may bedisplayed as a double frame (display example in FIG. 4E).

The present exemplary embodiment has been described by focusing on whichone of the right and left pupils is selected as the object to be broughtinto focus. Although not illustrated, the focus detection unit 155performs the focus detection processing based on an imaging signalcorresponding to the selected pupil. Then, the camera control unit 131transmits the focus-lens driving information (information about drivingof the focus lens or information similar thereto) calculated based onthe focus detection result obtained by the focus detection unit 155 tothe lens control unit 111. Accordingly, an appropriate pupil can bebrought into focus. A mode in which the camera control unit 131 controlsdriving of the focus lens group 104 to focus on the pupil designated bythe user in the present exemplary embodiment corresponds to a firstmode. A mode in which the camera control unit 131 controls driving ofthe focus lens group 104 to focus on the pupil selected by the automaticpupil selection processing corresponds to a second mode.

As described above, in a case where either the right or left pupil isselected, the designation state continues to be set while the detectionof the face corresponding to the pupil is continued. This makes itpossible to keep focusing on the pupil set by the user as a pupildesired for image capturing. In a case where the face becomes no longerdetected, it is considered that a scene originally intended by the userhas changed to a different scene. Thus, the designation of the pupil bythe user is canceled, and an automatically selected pupil is broughtinto focus. This makes it possible to keep focusing on the pupil byflexibly dealing with a change of the scene. Furthermore, since thedesignated pupil can always be selected, the pupil desired by the usercan always be brought into focus.

The present exemplary embodiment is described by using the example inwhich the pupil designation state is maintained while the detection ofthe face corresponding to the pupil is continued, and the pupildesignation is canceled by the user if the face becomes no longerdetected. However, here, a human body or a head may be used in place ofthe face. By using the human body or the head corresponding to thepupil, similar to the case of the face, it is possible to keep focusingon the intended pupil while the user can capture the intended object.

Next, a second exemplary embodiment of the present invention will bedescribed. In the present exemplary embodiment, either the right or leftpupil can be designated as an AF target object in a case where theobject is face authenticated by the user. This makes it possible tofocus on the same pupil in a case where the same person is capturedagain even if the user has temporarily lost track of the face or framinghas greatly changed.

The description of the configuration and the processing of the presentexemplary embodiment that are the same as those of the first exemplaryembodiment will be omitted where possible, and a difference will bemainly described. The second exemplary embodiment is different in thatthe face information detection unit 141 has a known face authenticationfunction. The face authentication function is a function of storing, ina memory (not illustrated), information (e.g., face feature pointinformation, also referred to as authentication information) foridentifying a face detected by the face information detection unit 141and subjected to a designating operation by the user. This enables, forexample, the authenticated face to be brought into focus preferentially.Here, in the present exemplary embodiment, information about which oneof the left and right pupils is selected by the user (also referred toas designation information) is stored in association with theauthentication information registered by the user. Accordingly, withrespect to the face authenticated by the face authentication function,the same pupil can be always brought into focus.

FIG. 5 illustrates a flowchart of operation in the camera control unit131 according to the second exemplary embodiment. In FIG. 5, thedescription of an operation the same as that in FIG. 2 will be omittedwhere possible, and an operation different from FIG. 2 will be mainlydescribed.

In step S501, the camera control unit 131 determines whether a face isdetected by the face information detection unit 141.

If the face is detected (YES in step S501), the operation proceeds tostep S502. In step S502, the camera control unit 131 determines whetherthe detected face is already authenticated by the face informationdetection unit 141.

If the detected face is not authenticated yet (NO in step S502), theoperation proceeds to step S503. In step S503, the camera control unit131 determines whether the face authentication has been designated forthe currently detected face. The user designates such faceauthentication by, for example, operating the touch panel 172 in a casewhere the user desires to authenticate the currently detected face.

If the face authentication has been designated (YES in step S503), theoperation proceeds to step S504. In step S504, the camera control unit131 controls the face information detection unit 141 to authenticate thedesignated face. The camera control unit 131 also controls the storageunit 153 to store the authentication information of the face.Furthermore, the camera control unit 131 sets an authenticated-stateflag.

If the detected face is already authenticated (YES in step S502), theoperation proceeds to step S505. In step S505, the camera control unit131 resets information for canceling the authentication. The informationfor canceling the authentication will be described in detail below.

Upon completion of a series of processes for authentication-relatedsetting in steps S502 to S505, the operation proceeds to step S506.Here, a flow of steps S506 to S517 and a flow of steps S519 to S521 aresimilar to the flow of steps S202 to S213 and the flow of steps S215 toS217 in FIG. 2 of the first exemplary embodiment, respectively. Thus,the flows will not be described. However, when the designationinformation of the pupil is stored in a memory (not illustrated) in stepS512, the camera control unit 131 performs control to store thedesignation information in association with the authenticationinformation stored in step S504.

If the face is not detected (NO in step S501), the operation proceeds tostep S518. In step S518, the camera control unit 131 performs control toexecute authentication state verification processing.

<Authentication State Verification Processing>

FIG. 6 is a diagram illustrating details of the authentication stateverification processing in step S518. The processing in step S518 andthe information for canceling the authentication will be described indetail with reference to FIG. 6.

In step S601, the camera control unit 131 determines whether the face iscurrently being authenticated. If the face is not currently beingauthenticated (NO in step S601), the operation proceeds to step S605. Instep S605, the camera control unit 131 clears the pupil designatingflag, as with the first exemplary embodiment. On the other hand, if theface is currently being authenticated (YES in step S601), the operationproceeds to step S602. In and after step S602, the camera control unit131 performs control to determine whether to cancel the authentication.Here, as an example, setting is made to cancel the authentication in acase where a predetermined time has elapsed since the user has losttrack of the face. In other words, the camera control unit 131increments the counter in step S602 each time the face becomes no longerdetected and the operation proceeds to the processing in FIG. 6 (stepS518 in FIG. 5). Then, in step S603, the camera control unit 131determines whether the value of the counter is less than a threshold. Ifthe value of the counter is less than the threshold (YES in step S603),the processing of the flow ends. If the value of the counter is thethreshold or more (NO in step S603), the operation proceeds to stepS604. In step S604, the camera control unit 131 clears theauthenticated-state flag. If the face is not detected again even afterthe predetermined time has elapsed, the authentication and the pupildesignation are both canceled simultaneously by this processing. In thisexample, the above-described information for canceling theauthentication is equivalent to the counter value. If the authenticatedface is found again before the counter value reaches the threshold, thecurrent state can be returned to an initial state by resetting thecounter.

In a case where a long time has elapsed since the previous detection ofthe face, it is highly possible that the captured scene has greatlychanged and the image-capturing target has changed. Thus, theauthentication state can be automatically canceled by executing thisprocessing.

It is also possible to always designate the same pupil every time theauthenticated person is detected by enabling the user to set thethreshold of the counter and by setting the threshold to infinity.

Furthermore, in the present exemplary embodiment, there is provided theconfiguration in which the face is authenticated based on the setting bythe user. However, other configuration may also be used. For example,the detected face may be automatically authenticated, and theauthentication result may be made valid later in a case where the pupildesignation is performed. As described above, necessary detectionoperation can be performed by using the authentication function, withoutundermining the intention of the user.

The present invention has been described in detail above with referenceto the two exemplary embodiments. However, the present invention is notlimited to these specific exemplary embodiments, and includes variousforms in a range not departing from the gist of the present invention.The above-described exemplary embodiments may also be partially combinedas appropriate.

The present invention also includes a case where a program of softwarethat implements the functions of the above-described exemplaryembodiments is supplied to a system or apparatus that has a computercapable of executing the program. The program is supplied to the systemor apparatus directly from a recording medium or by using wired/wirelesscommunication. Therefore, the present invention is also implemented by aprogram code that is supplied to and installed on the computer toimplement the functions and processing of the present invention usingthe computer. In other words, the computer program for implementing thefunctions and processing of the present invention is also included inthe present invention. In this case, the program may be in any form suchas an object code, a program to be executed by an interpreter, andscript data to be supplied to an operating system as long as a functionas the program is provided. Examples of the recording medium forsupplying the program include a hard disk, a magnetic recording mediumsuch as a magnetic tape, an optical/magneto-optical storage medium, anda nonvolatile semiconductor memory. A method for supplying the programmay be such a method that a computer program that forms the presentinvention is stored in a server on a computer network, and a clientcomputer connects to the server to download the computer program andexecutes the downloaded computer program.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2019-007011, filed Jan. 18, 2019, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing apparatus comprising at leastone processor configured to perform operation of: a first detection unitconfigured to detect a first area based on an image signal output froman image sensor; a second detection unit configured to detect a secondarea corresponding to a first pupil included in a face, the second areabeing included in the first area; a designation unit configured todesignate the second area detected by the second detection unit; and acontrol unit configured to control driving of a focus lens based on animage signal corresponding to the second area, wherein the designationunit cancels designation of the second area in a case where the firstarea becomes no longer detected by the first detection unit.
 2. Theimaging apparatus according to claim 1, wherein in a case wheredetection of the first area is continued by the first detection unit,the designation unit continues designation of the second area even in acase where the second area becomes no longer detected by the seconddetection unit.
 3. The imaging apparatus according to claim 2, whereinin a case where the detection of the first area is continued by thefirst detection unit and where the second area is detected by the seconddetection unit after the second area becomes no longer detected by thesecond detection unit, the control unit controls driving of the focuslens based on an image signal corresponding to the second area.
 4. Theimaging apparatus according to claim 1, wherein in a case wheredetection of the first area is continued by the first detection unit andwhere the second area becomes no longer detected by the second detectionunit, the control unit drives the focus lens based on an image signalcorresponding to the first area.
 5. The imaging apparatus according toclaim 1, further comprising a selection unit configured to select eitherthe second area detected from the first area by the second detectionunit or a third area corresponding to a second pupil different from thefirst pupil, wherein the control unit has a first mode to drive thefocus lens based on an image signal corresponding to the second areadesignated by the designation unit, and a second mode to control drivingof the focus lens based on an image signal corresponding to the secondarea or the third area selected by the selection unit, and wherein thecontrol unit shifts from the first mode to the second mode in a casewhere designation of the second area is canceled by the designationunit.
 6. The imaging apparatus according to claim 1, further comprisinga storage unit configured to store designation information of a userabout a face registered by the user and a pupil included in the face,wherein even in a case where the first area becomes no longer detectedby the first detection unit, the designation unit does not canceldesignation of the first pupil in a case where the first area is theface registered by the user.
 7. The imaging apparatus according to claim1, wherein the first area is an area corresponding to a face.
 8. Theimaging apparatus according to claim 1, wherein the designation unitcancels designation of second area in a case where the first area isdesignated by a user by touch operation and the first area becomes nolonger detected by the first detection unit for a predetermined timeperiod.
 9. The imaging apparatus according to claim 8, wherein thedesignation unit continues designation of the second area in a casewhere the first area is designated by the user by the touch operationand the first area becomes no longer detected by the first detectionunit for a shorter time than the predetermined time.
 10. The imagingapparatus according to claim 1, wherein the designation unit continuesdesignation of the second area in a case where the first area isdesignated by the user by the touch operation and the first area becomesno longer detected by the first detection unit for a shorter time thanthe predetermined time.
 11. A control method for an imaging apparatus,the control method comprising: detecting a first area based on an imagesignal output from an image sensor; detecting a second areacorresponding to a first pupil included in a face, the second area beingincluded in the first area; designating the detected second area; andcontrolling driving of a focus lens based on an image signalcorresponding to the second area, wherein designation of the second areais canceled in a case where the first area becomes no longer detected.